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1.  Introduction 


The  health  and  safety  electromagnetic  fields  (EMF)  standards  in  the  radio  frequency  (RF)  range  are 
primarily  based  on  the  concept  of  a  thermal  mechanism  and  protect  against  "thermal"  effects. 
Indeed,  the  current  consensus  is  that  only  thermal  RF  effects  could  be  harmful,  but  this  certainly 
does  not  mean  that  studies  reporting  possible  non-thermal  effects  were  ignored.  This  consensus  is 
based  on  rigorous  analysis  of  published  studies,  which  have  to  satisfy  such  strict  criteria  as 
replication  in  several  species,  under  different  field  conditions,  and  that  the  effects  could  be 
considered  potentially  harmful  in  humans.  A  brief  review  of  some  RF  safety  standards,  such  as  the 
International  Commission  of  Non-ionizing  Radiation  Protection  (ICNIRP)  and  Institute  of  Electrical 
and  Electronics  Engineers,  Inc.  (IEEE)  standards  (ICNIRP  2009;  IEEE  2006),  reveals  that  they  include 
consideration  of  frequency  dependent  absorption  in  human  body  (whole  body  resonance).  In 
addition,  special  restrictions  on  localized  exposure  including  methods  for  volume  averaging  are 
introduced.  These  documents  specify  time-averaged  whole-body-averaged  SAR  values  and  peak 
spatial-average  SAR  values,  neither  of  which  should  be  exceeded.  The  spatial  peak  SAR  is  usually 
averaged  over  a  specified  volume  of  10  grams  (contiguous  tissue  (ICNIRP  2009)).  Current  RF 
exposure  standards  are  generally  derived  from  an  assumption  of  uniform  field  exposure  of  the 
entire  body.  While  the  fundamental  basis  of  these  standards  is  related  to  limiting  the  SAR  in  the 
body,  as  averaged  over  the  entire  body  mass,  most  standards,  such  as  those  of  the  (IEEE  2006)  and 
(ICNIRP  2009),  also  contain  Maximum  Permissible  Exposure  (MPE)  limits  and  reference  levels 
related  to  peak  SAR  that  may  occur  at  any  point  within  the  body,  averaged  over  10  grams  of  tissue. 
These  spatial  peak  SAR  limits  are  derived  from  observations  of  uniform  exposure  of  laboratory 
animals  or  phantom  models  of  both  animals  and  humans.  In  recognition  of  the  non-uniform 
absorption  of  RF  energy  within  the  body,  even  with  uniform  field  exposure  over  the  body,  the 
standards  set  a  limit  on  the  spatial  peak  SAR  at  25  times  the  whole-body  average  value.  For 
example,  while  the  whole-body  average  SAR  is  limited  to  0.4  watts  per  kilogram  (W/kg)  in  the 
whole  body,  a  local  SAR  of  10  W/kg,  averaged  over  10  grams  of  tissue,  is  permitted.  For 
determining  SAR  values  inside  the  human  body,  two  approaches  have  been  established.  The  first  is 
measurement  in  body-simulating  liquid  filled  phantoms,  while  the  second  is  numerical  dosimetry. 
For  measurements,  the  liquid  represents  a  homogeneous  body  with  a  constant  conductivity  and 
relative  permittivity.  Numerical  simulations  can  on  the  other  hand  provide  the  possibility  of 
investigating  exposure  of  specific  organs  with  more  realistic  tissue  and  geometric  properties. 


There  are  several  methods  for  simulating  the  interaction  of  electromagnetic  fields  with  the  body, 
based  on  different  numerical  approaches  (Hand  2008);  finite-difference  time-domain,  in  short  FDTD 
is  among  the  most  commonly  used  for  dosimetric  applications,  due  to  its  robustness,  relative 
simplicity  of  implementation  and  favorable  scaling  of  computing  cost  versus  problem  size.  FDTD  is  a 
numerical  method  for  solving  full-wave  Maxwell  equations  in  settings  with  complex  geometries 
and/or  incident  electromagnetic  fields  (Talfove  and  Hagness  2005).  The  method  was  pioneered  by 
Yee  in  1966,  who  devised  an  algorithm  where  the  E  and  H  fields  are  computed  in  separate  steps 
using  second-order  finite  difference  approximation  of  the  Maxwell  equations  (Yee  1966).  The  FDTD 
algorithm  lends  itself  to  very  efficient  parallelization,  thus  enabling  the  use  of  supercomputers, 
cluster  computing  and  recently  also  parallel  computing  on  graphics  processing  units  (Inman  and 
Elsherbeni  2005). 

The  main  objective  of  this  project  is  to  verify  the  parallel  C  code  program  package  for  FDTD 
calculations  of  human  exposure  due  to  the  electromagnetic  fields  (EMF)  which  was  developed  by 
Radio  Frequency  Radiation  Branch,  AFRL,  Brooks  City-Base,  Texas  with  parallel  Fortran  code  which 
gained  wide  confidence  in  the  results  among  the  scientists  (Gajsek  et  al.  2002;  Gajsek  et  al.  2001). 
The  older  algorithm  written  in  Fortran  and  based  on  LAM/MPI  clustering  and  message  parsing 
platform  in  Linux  was  used  as  a  benchmark  for  verification.  The  second  implementation  is  written  in 
C  and  runs  on  OpenMPI  clustering  platform,  also  in  Linux.  Both  platforms  were  also  compared  with 
third-party  commercial  FDTD  software  package  SEMCAD  X  (Speag,  Zurich,  Switzerland). 

With  this  effort  AFRL  and  INIS  could  fully  compare  the  existing  methods  and  equipment  for  SAR 
predictions  in  various  digital  numerical  models  and,  thus,  increase  the  confidence  in  numerical 
dosimetry  results.  An  error  margin  will  be  analyzed  and  fully  described  that  is  due  to  computer 
hardware  and  software  tools  and  different  programming  language. 


2.  Protocol  definition 

To  provide  a  framework  for  the  verification,  a  protocol  was  defined,  before  the  verification  was 
performed.  The  protocol  was  defined  in  order  to  provide  a  framework  for  comparing  the  two 
different  versions  of  FDTD  code.  A  range  of  different  models  and  exposure  scenarios  was  chosen  to 
represent  typical  uses  of  the  software.  The  models  chosen  for  the  comparison  were  the  1mm  man 
(1878  by  340  by  584  voxels),  the  2mm  man  (939  by  179  by  293  voxels),  the  3mm  man  (196  by  114 
by  626  voxels)  and  a  rat  model  (51  by  22  by  114  voxels). 


A  computer-segmented  set  of  the  photographic  images  was  created  by  National  University  of 
Singapore  and  Johns  Hopkins  University.  Each  of  the  1878  slices  in  the  XY  plane  was  then  more 
segmented  with  a  palette  of  colors  that  represented  the  40  tissue  types  (the  method  of 
construction  is  described  in  (PA  Mason  et  al.  1995)).  Each  voxel  is  a  cube  1  mm  on  a  side  and  the 
model  consists  of  374  million  voxels  (1878  x  340  x  586  voxels).  Predicting  SAR  values  in  this  1  mm^ 
whole-body  model  using  FDTD  requires  a  minimum  of  18  GB  of  computer  memory.  Since  this  high- 
resolution  model  requires  a  substantial  amount  of  computer  memory,  smaller  versions  of  this 
dataset  have  been  created  and  are  suitable  for  some  applications. 

These  reduced  resolution  (2  mm  and  3  mm)  models  were  created  automatically.  The  process  was 
as  follows  for  creating  a  3-mm  anatomical  model  from  a  1-mm  model.  Layers  of  air  were  added  to 
one  or  more  sides  of  the  model  volume  to  make  the  size  of  the  model  an  even  multiple  of  the  3 
mm^.  The  reduction  then  takes  a  cube  of  3  by  3  by  3  one-millimeter  voxels  and  based  on  the  most 
common  type  in  that  cube  creates  the  single  3  mm  voxel.  This  process  was  repeated  for  each  3  by  3 
by  3  set  of  1-mm  voxels. 

The  models  were  used  to  calculate  the  SAR  distribution  in  a  planewave  exposure  in  two  different 
incident  field  orientations,  MEHK  (0=90°,  4)=270°,  Eince=l,  Einc())=0,  Epeak=l  V/m)  and  PEHK  (0=90°, 
4)=90°,  Einc0=l,  E|nc())=0,  Epeak=l  V/m),  within  a  frequency  range  from  70  to  2000  MHz.  The  criteria  for 
comparison  were  the  maximum  tissue  SAR  for  all  tissues,  the  whole  body  SAR  for  each  model  and 
frequency  and  SAR  in  the  vertical  plane  through  the  centre  of  the  model.  The  difference  in  each 
data  point  should  be  less  than  ±  10  %. 

FDTD  code  is  well  known  and  widespread  numerical  modeling  technique  to  calculate  field 
distribution  inside  the  human  exposed  to  electromagnetic  field.  In  Brooks  AFRL  the  code  was 
developed  about  10  years  ago  and  lately  intensively  used  in  the  filed  of  numerical  dosimetry.  The 
original  code,  which  is  freely  available,  is  written  in  Fortran.  It  was  used  as  a  base  for  some  of  the 
commercially  available  program  packages  for  FDTD  calculation. 

Since  in  the  last  few  years  C  programming  language  is  becoming  more  popular  than  the  Fortran  the 
FDTD  code  was  rewritten  in  the  C.  The  old  Fortran  FDTD  code  was  widely  accepted  and  used  as  a 
suitable  method  for  numerical  calculations  of  electromagnetic  field  exposures.  Before  a  wide  use  of 
the  new  C  code  verification  is  needed  for  this  code.  Verification  is  a  process  which  requires 
parametric  analysis  and  calculations  of  SAR  values  for  different  voxel  models.  By  comparing  the 
results  of  a  set  of  calculations  with  the  Fortran  code  with  the  same  set  of  calculations  with  the  C 
code  we  will  validate  the  C  code.  It  is  important  to  plane  the  set  of  the  calculations  to  minimize  the 
possibility  to  overlook  errors  or  faults  in  the  C  code. 


Additionally  the  evaluation  of  the  effect  of  number  of  PML  and  air  layers  on  the  SAR  values  was 
investigated,  and  the  speed  and  parallel  efficiency  of  calculation  were  compared  between  the  two 
implementations  of  the  FDTD  algorithm,  as  well  as  a  comparison  with  the  third-party  SEMCAD  X 
software. 

For  the  calculations  a  cluster  of  multi  processor  computers  is  used.  It  consists  of  two  multi 
processor  computers,  in  each  computer  2  quad  core  Xeon  processors  at  1.6  GFIz  and  12  GB  RAM  is 
installed,  altogether  16  cores  and  24  GB  of  RAM  is  available. 

There  is  Centos  5  installed  on  the  computers  as  well  as  the  Intel  Fortran  9  compiler  and  LAM-MPI. 
On  the  cluster,  parallel  Fortran  version  of  Brooks  FDTD  code  is  running. 


3.  Results 


[r^ 


1  mm  man  model 

This  was  the  highest  resolution  and  the  largest  model  used  in  the  calculation.  The  result  was 
performed  at  only  one  frequency  -  2000  MHz,  with  the  problem  size  being  approximately  447 
Mcells.  The  incident  field  orientation  was  MEHK  (9=90°,  (^=270°,  Eir,ce=l,  Einc(j)=0,  Epeak=l  V/m).  The 
total  computation  time  was  48  hours  and  50  minutes,  the  number  of  time-steps  in  the  simulation 
was  2080.  This  gives  an  average  calculation  speed  of  5.3  Mcells/second. 

The  results  of  the  1mm  model  show  a  9  %  difference  in  results  between  the  C  and  Fortran  codes  for 
the  whole  body  SAR,  as  shown  in  Table  1. 

Frequency  [MHz]  Fortran  [W/kg]  C  [W/kg]  %  difference 
100  0.051812  0.047322  -9% 

Table  1:  Results  of  the  1mm  model  in  the  Fortran  and  C  codes 


The  mean  normalized  difference  of  the  maximum  tissue  SAR  values  (shown  in  Table  2)  is  -29  %, 
while  the  mean  normalized  difference  of  the  mean  tissue  SAR  is  -25  %.  If  the  values  for  the 
normalized  difference  are  weighted  with  the  number  of  voxels  in  each  tissue,  the  values  are  -0.5  % 
and  -0.2  %  for  maximum  and  mean  tissue  SAR,  respectively.  Tissue  SAR  values  correspond  in  all 
mentioned  cases  to  the  maximum  (single  voxel)  SAR  in  the  tissue,  or  the  mean  tissue  SAR,  i.e.  the 
average  of  SAR  values  over  all  voxels  in  the  tissue. 


tissue 


C  code 


Fortran  code 


max  sar 

[W/kg] 

mean  sar 

[W/kg] 

max  sar 

[W/kg] 

mean  sar 

[W/kg] 

BILE 

0.006099 

0.001566 

0.21235 

0.041979 

BODY  FLUID 

0.993493 

0.012824 

2.9976 

0.046456 

EYE  (cornea) 

0.042668 

0.013682 

0.93334 

0.29773 

FAT 

3.89667 

0.021974 

10.484 

0.020427 

LYMPH 

MUSCOUS 

0.416686 

0.001601 

1.8877 

0.021414 

MEMBRANE 

0.352977 

0.003615 

4.361 

0.055832 

NAILS  (toe  &  finger) 

0.817338 

0.077769 

1.7624 

0.20442 

NERVE  (spine) 

2.548601 

0.027095 

0.74659 

0.014396 

MUSCLE 

8.617774 

0.049307 

6.5786 

0.048736 

HEART 

0.01187 

0.001232 

0.12807 

0.005391 

WHITE  MATTER 

0.323376 

0.026388 

0.19767 

0.016669 

STOMACH 

0.023238 

0.001423 

0.76467 

0.012716 

GLANDS 

1.002455 

0.010366 

1.6377 

0.077203 

BLOODVESSEL 

5.64557 

0.038411 

8.5819 

0.076469 

LIVER 

0.141912 

0.002211 

0.3282 

0.005453 

GALLBLADDER 

0.008603 

0.001348 

0.43787 

0.025449 

SPLEEN 

0.113244 

0.005189 

0.048623 

0.001684 

CEREBELLUM 

0.162809 

0.018788 

0.038388 

0.002358 

BONE  (cortical) 

1.168127 

0.011145 

2.358 

0.027413 

CARTILAGE 

2.85327 

0.015508 

10.071 

0.081094 

LIGAMENTS 

3.761534 

0.053455 

5.8432 

0.10828 

SKIN/DERMIS 

10.53239 

0.298725 

39.166 

0.3207 

INTESTINE  (large) 

0.479863 

0.003542 

1.0679 

0.026054 

TOOTH 

0.008644 

0.000684 

0.4391 

0.029196 

GRAY  MATTER 

0.421018 

0.034685 

0.61099 

0.02381 

EYE  (lens) 

0.029349 

0.008608 

0.85487 

0.27268 

LUNG  (outer) 

0.094155 

0.003409 

0.25997 

0.011928 

INTESTINE  (small) 

0.066383 

0.00205 

1.1012 

0.019203 

EYE  (sclera/wall) 

0.084886 

0.010594 

2.0483 

0.21677 

LUNG  (inner) 

0.253398 

0.00549 

1.014 

0.014397 

PANCREAS 

0.010921 

0.001199 

0.006803 

0.001053 

BLOOD 

CEREBRAL  SPINAL 

7.328976 

0.013444 

7.0858 

0.037708 

FLUl 

0.692121 

0.053989 

0.80209 

0.035709 

EYE  (aqueous  humor) 

0.071214 

0.019881 

1.1122 

0.33588 

KIDNEYS 

0.02343 

0.001701 

0.007199 

0.001126 

BONE  MARROW 

0.534079 

0.007642 

1.4257 

0.015365 

BLADDER 

0.014046 

0.000656 

3.1387 

0.01044 

TESTICLES 

0.003193 

0.001209 

1.1866 

0.25478 

BONE  (cancellous) 

1.391796 

0.017978 

1.8113 

0.032798 

Table  2:  Tissue  maximum  and  mean  SAR  in  the  1mm  man  model. 


2  mm  man  model 


The  2  mm  man  was  run  in  a  frequency  range  from  100  to  2000  MHz,  with  a  step  of  100  MHz,  in  the 
MEHK  (0=90°,  4)=270°,  Einc0=l,  Einc(j)=0,  Epeak=l  V/m)  incident  field  orientation.  The  values  of  whole- 
body  SAR  are  presented  in  Table  3. 


Frequency  [MHz] 

Fortran  [W/kg] 

C  [W/kg] 

%  difference 

100 

0.113036 

0.111546 

-1% 

200 

0.053451 

0.069123 

29% 

300 

0.056834 

0.057689 

2% 

400 

0.063927 

0.06077 

-5% 

500 

0.065186 

0.061973 

-5% 

600 

0.065161 

0.062287 

-4% 

700 

0.063500 

0.061336 

-3% 

800 

0.063326 

0.060706 

-4% 

900 

0.062653 

0.060285 

-4% 

1000 

0.062618 

0.061052 

-3% 

1100 

0.061655 

0.059845 

-3% 

1200 

0.061946 

0.059239 

-4% 

1300 

0.062763 

0.059063 

-6% 

1400 

0.062146 

0.058634 

-6% 

1500 

0.060409 

0.058475 

-3% 

1600 

0.058856 

0.057524 

-2% 

1700 

0.057357 

0.056685 

-1% 

1800 

0.056709 

0.055575 

-2% 

1900 

0.055952 

0.054456 

-3% 

2000 

0.055794 

0.053205 

-5% 

Table  3:  Whole-body  SAR  values  for  the  2mm  man  model 

The  values  of  whole-body  SAR  versus  frequency  are  plotted  in  Figure  1.  In  all  the  data  points  but 
one,  the  whole-body  SAR  from  the  two  different  versions  of  the  code  does  not  differ  by  more  than 
6  %,  with  the  Fortran  code  generally  producing  slightly  higher  values  than  the  C  code.  A  statistical 
analysis  was  performed  on  the  two  sets  of  data  to  compare  the  results  of  the  two  different  codes. 
The  difference  between  each  case  at  every  frequency  was  compared  to  determine  if  there  was  any 
statistically  significant  difference  between  the  results.  In  each  case,  a  Wilcoxon  signed  rank  test 
was  used,  since  the  results  were  not  distributed  normally  (Shapiro-Wilk  test  for  normality  (P  < 
0.050)).  The  non-normality  of  the  distribution  of  results  prevented  the  use  of  a  paired  t-test.  The 
results  of  the  analysis  show,  that  for  the  2  mm  man,  there  is  a  significant  difference  between  the 
two  different  codes  (P=0.001). 
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Figure  1:  Whole  body  SAR  in  2mm  man  model  vs.  frequency. 

Tissue/organ  maximum  and  mean  SAR  values  were  also  compared.  The  largest  differences  for 
maximum  SAR  were  252  %  (all  percentages  were  calculated  with  the  formula  (SARc-SARf)/SARf)  in 
the  tissue  "stomach"  at  100  MHz,  and  -75  %  in  the  tissue  "kidneys"  at  1600  MHz.  The  average  error 
was  8  %,  with  the  standard  deviation  of  the  error  being  35  %.  These  values  are  much  smaller  for 
mean  tissue  SAR,  which  is  expected,  since  these  are  representative  of  the  whole  tissue  mass,  which 
can  range  from  a  less  than  a  gramm  to  several  kilograms  (from  42  cells,  or  less  than  a  gramm  for 
tissue  "bile",  to  more  than  5  million  cells  and  more  than  40  kilograms  for  tissue  "muscle").  The 
maximum  difference  in  the  mean  tissue  SAR  were  81  %  in  the  tissue  "lymph"  at  200  MHz,  and  -55% 
in  the  tissue  "spleen"  at  1200  MHz.  The  mean  difference  was  1  %  while  the  standard  deviation  was 
15  %. 
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Figure  2:  Maximum  tissue  SAR  vaiuesfor  tissues  "biie",  "body  fluid"  and  "eye  -  cornea" 


Figure  3:  Maximum  tissue  SAR  vaiuesfor  tissues  "fat",  "iymph"  and  "mucous  membrane" 
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Figure  4:  Maximum  tissue  SAR  vaiuesfor  tissues  "naiis",  "nerve"  and  "muscie" 


Figure  5:  Maximum  tissue  SAR  vaiuesfor  tissues  "heart",  "white  matter"  and  "stomach" 
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Figure  6:  Maximum  tissue  SAR  vaiuesfor  tissues  "glands",  "biood  vessei"  and  "iiver" 
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Figure  7:  Maximum  tissue  SAR  vaiuesfor  tissues  "galibladder",  "spieen"  and  "cerebeiium" 
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Figure  8:  Maximum  tissue  SAR  vaiuesfor  tissues  "bone  -  corticai",  "cartiiage"  and  "iigaments" 
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Figure  9:  Maximum  tissue  SAR  vaiuesfor  tissues  "skin",  "intestine"  and  "tooth" 
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Figure  10:  Maximum  tissue  SAR  vaiuesfor  tissues  "gray  matter",  "eye  -  iens"  and  "iung" 


Figure  11:  Maximum  tissue  SAR  vaiuesfor  tissues  "intestine",  "eye  -  sciera"  and  "iung  -  inner" 
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Figure  12:  Maximum  tissue  SAR  vaiuesfor  tissues  "pancreas",  "blood"  and  "cerebrospinal  fluid" 


Figure  13:  Maximum  tissue  SAR  values  for  tissues  "eye  -  agueous  humor",  "kidneys"  and  "bone 
marrow" 
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Figure  14:  Maximum  tissue  SAR  vaiuesfor  tissues  "biadder",  "testicies"  and  "bone  -  canceiious" 


Figures  2-14  above  show  the  frequency  dependency  of  each  different  tissue  type  as  calculated  by 
the  two  different  FDTD  implementations.  The  graphs  show  a  good  agreement  in  tissue  SAR  values 
across  a  very  broad  range  of  frequencies  and  different  tissue  permittivities  and  conductivities. 


CQ 

g 

QC 

< 

CO 


\— 

DC 

< 

CO 

o 

re 

DC 


0  500  1000  1500  2000 

Frequency  [MHz] 


Figure  15:  Ratio  between  maximum  tissue  SAR  and  whole-body  SAR 


Figure  16:  SAR  in  the  middie  of  the  body  in  the  2  mm  man.  Left:  resuit  of  C  code.  Right:  resuit  of 
Fortran  code. 


The  ratio  of  tissue  maximum  SAR  /  whole-body  SAR  versus  frequency  is  shown  in  Figure  15. 
Although  the  use  of  tissue  maximum  SAR  instead  of  an  average  over  10  g  of  tissue  was  used,  which 
yields  a  significantly  more  conservative  results,  the  maximum  SAR  /  whole-body  SAR  ratio  is  lower 
than  the  25  recommended  by  the  international  guidelines  and  standards  in  all  investigated  tissues 
(Heart,  Grey  matter.  White  matter.  Eye  lens  and  Nerve),  at  frequencies  above  800  MHz.  It  is  higher 
than  25  only  in  the  tissue  Nerve  at  lower  frequencies  which  may  be  the  result  of  a  few  single  voxels 
with  very  high  values  of  SAR. 


3  mm  man  model 


The  two  codes  were  compared  in  a  total  of  20  points,  of  which  10  were  in  the  MEHK  (0=90°, 
4)=270°,  Eince=l,  Einc<j,=0,  Epeak=l  V/m)  and  10  in  the  PEHK  (0=90°,  4)=90°,  Eince=l,  Einc<j,=0,  Epeak=l  V/m) 
incident  field  configuration. 


orientation 

Frequency 

[MHz] 

Fortran  [W/kg] 

C  [W/kg] 

% 

difference 

200 

0.05352 

0.06938 

30% 

400 

0.06362 

0.06122 

-4% 

600 

0.06498 

0.06243 

-4% 

800 

0.0633 

0.06106 

-4% 

MEHK 

1000 

0.06243 

0.06163 

-1% 

1200 

0.06087 

0.0604 

-1% 

1400 

0.06125 

0.06121 

0% 

1600 

0.06035 

0.06041 

0% 

1800 

0.0588 

0.05908 

0% 

2000 

0.05737 

0.05732 

0% 

200 

0.04888 

0.06447 

32% 

400 

0.06158 

0.05899 

-4% 

600 

0.06702 

0.06357 

-5% 

800 

0.06437 

0.06312 

-2% 

PEHK 

1000 

0.06347 

0.06313 

-1% 

1200 

0.06176 

0.06147 

0% 

1400 

0.06053 

0.06071 

0% 

1600 

0.05842 

0.05855 

0% 

1800 

0.05684 

0.05656 

0% 

2000 

0.05542 

0.05548 

0% 

Table  4:  Whole  body  SAR  results  for  the  3  mm  man  model 

The  results  are  presented  in  Table  4.  All  results  above  200  MHz  show  very  good  agreement 
between  the  two  different  codes,  while  the  frequencies  below  show  slightly  larger  errors.  The 
largest  difference  of  32  %  was  found  at  200  MHz.  The  values  of  whole-body  SAR  versus  frequency 
are  shown  also  in  Figure  17  below. 
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Figure  17:  Whole  body  SAR  in  3mm  man  model  vs.  frequency 


4.  The  role  of  boundary  conditions 

A  statistical  analysis  was  performed  on  the  two  sets  of  data  to  compare  the  results  of  the  two 
different  codes.  The  difference  between  each  case  at  every  frequency  was  compared  to  determine 
if  there  was  any  statistically  significant  difference  between  the  results.  In  each  case,  a  Wilcoxon 
signed  rank  test  was  used,  since  the  results  were  not  distributed  normally  (Shapiro-Wilk  test  for 
normality  (P  <  0.050)).  The  non-normality  of  the  distribution  of  results  prevented  the  use  of  a 
paired  t-test.  The  results  of  the  analysis  show,  that  for  the  3mm  model,  no  significant  difference 
was  observed  in  the  results  (P=l). 

The  3  mm  model  was  also  used  to  evaluate  the  effect  of  varying  the  number  of  air  layers  and  PML 
layers.  The  results  are  shown  in  Figures  18  and  19. 
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Figure  18:  Influence  of  the  number  of  PML  layers  on  the  final  whole-body  SAR  results 
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Figure  19:  Influence  of  the  number  of  air  layers  on  the  final  whole-body  SAR  results 


The  results  of  varying  the  number  of  air  and  PML  layers  show,  that  these  parameters  have  a  minor 
effect  on  the  results,  as  long  as  they  are  varied  within  reasonable  limits.  Figures  18  and  19  show, 
that  the  values  of  whole-body  SAR  change  by  less  than  1.1  %  in  the  whole  range  of  tested  cases. 


The  efficiency  of  clustering  was  evaluated  and  found  to  be  very  close  to  the  theoretical  limit  of 
parallelization  (Figure  20).  The  decrease  of  time  following  the  ideal  curve  which  would  follow  the 
equation  Tn=T]/n,  where  n  is  the  number  of  computational  nodes,  Tj  is  the  time  needed  to  solve  the 
problem  on  a  single  node  and  T„  is  the  time  needed  to  solve  the  problem  on  n  nodes. 
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Figure  20:  Time  required  for  solving  a  model  versus  the  number  of  computational  nodes. 


5.  Comparison  with  third-party  FDTD  software  SEMCAD  X 

The  results  of  the  two  different  code  implementations  were  compared  with  the  third  party  FDTD 
software  package  SEMCAD  X,  version  14.2.1  (SPEAG,  Zurich,  Switzerland).  The  values  of  whole-body 
SAR  and  tissue  SAR  were  compared  at  two  frequencies:  1.5  and  2  GHz  (Table  5).  The  model  used  a 
2mm  voxel  resolution  and  they  all  used  the  same  incident  field  orientation.  The  model  in  the 
SEMCAD  software  is  shown  in  Figure  21. 


Figure  21 :  The  orientation  of  the  modei  in  the  coordinate  system  and  the  iocation  of  the  pianewave 
field  source  (red  wire  box). 


At  1.5  GHz  the  results  for  whole  body  SAR  were  0.049  W/kg,  0.058  W/kg  and  0.60  W/kg  in  SEMCAD, 
C  and  Fortran  codes,  respectively.  At  2  GHz  these  results  were:  0.047  W/kg,  0.053  W/kg  and  0.056 
W/kg  in  SEMCAD,  C  and  Fortran  codes,  respectively.  This  shows  that  the  SEMCAD  software  package 
produces  significantly  lower  results,  but  these  may  be  due  to  differences  in  the  SAR  averaging  and 
statistics  implementations.  Also  the  the  SEMCAD  software  does  not  automatically  ensure  that  a 


steady  state  is  reached  in  the  model,  as  it  doesn't  compute  SAR  during  the  simulation,  but  only 
during  post-processing. 


The  results  of  comparison  of  single  tissues  are  consistent  with  the  results  for  whole-body  SAR  in 
that  the  SEMCAD  software  produces  values  that  are  consistently  lower  than  the  two  Brooks  parallel 
FDTD  codes. 


Tissue  Mean  SAR 
1500  MHz  2000  MHz 

Tissue  SEMCAD  C  SEMCAD  C 


Bile 

0.069 

0.086 

0.030 

0.037 

Bladder 

0.008 

0.011 

0.007 

0.011 

Blood 

0.021 

0.032 

0.022 

0.034 

Blood_vessel 

0.079 

0.057 

0.053 

0.065 

BodyFluid 

0.084 

0.066 

0.044 

0.049 

Bone 

0.019 

0.028 

0.022 

0.029 

Bone_cancellous 

0.031 

0.036 

0.035 

0.037 

Brain_grey_matter 

0.045 

0.065 

0.030 

0.036 

Brain_white_matter 

0.029 

0.047 

0.018 

0.025 

Cartilage 

0.092 

0.110 

0.086 

0.094 

Cerebellum 

0.003 

0.007 

0.002 

0.005 

Cerebrospinal_fluid 

0.044 

0.065 

0.028 

0.037 

Cornea 

0.156 

0.157 

0.413 

0.512 

Eyejens 

0.069 

0.132 

0.208 

0.227 

Eye_sclera 

0.159 

0.143 

0.223 

0.239 

Eye_vitreous_humor 

0.219 

0.207 

0.303 

0.335 

Fat 

0.024 

0.021 

0.015 

0.021 

GallBladder 

0.050 

0.041 

0.020 

0.025 

Glands 

0.077 

0.087 

0.083 

0.088 

Heart_muscle 

0.012 

0.016 

0.004 

0.010 

Largejntestine 

0.049 

0.054 

0.024 

0.029 

Liver 

0.008 

0.013 

0.004 

0.009 

Lung 

0.012 

0.043 

0.007 

0.026 

Lymph 

0.033 

0.032 

0.018 

0.022 

Marrow_red 

0.044 

0.015 

0.012 

0.014 

Mucosa 

0.079 

0.086 

0.062 

0.066 

Muscle 

0.051 

0.064 

0.043 

0.052 

Nails 

0.070 

0.119 

0.107 

0.237 

Nerve 

0.015 

0.021 

0.011 

0.014 

Pancreas 

0.002 

0.004 

0.000 

0.004 

Skin 

0.248 

0.326 

0.347 

0.339 

SmalIJntestine 

0.033 

0.038 

0.016 

0.022 

Spleen 

0.001 

0.004 

0.001 

0.005 

Stomach 

0.032 

0.035 

0.013 

0.020 

Teeth 

0.044 

0.052 

0.033 

0.032 

Tendon_Ligament 

0.096 

0.102 

0.107 

0.107 

Testis 

0.177 

0.175 

0.232 

0.262 

Total  Selected 

0.049 

0.058 

0.047 

0.053 

Table  5:  Tissue  maximum  and  mean  SAR  values  in  the  SEMCAD  and  C  code  results 


Figure  22:  SAR  in  a  vertical  cross-section  of  the  SEMCAD  model. 


The  SAR  values  in  Figure  22  were  normalized  to  the  same  incident  field  power  density  (ImW/cm^) 
as  in  the  Brooks  codes  (Figure  16).  A  side-by-side  comparison  is  made  in  Figure  23. 


Figure  23:  Slice  showing  the  SAR  distribution  in  the  three  different  software  packages.  From  left  to 
right  are  a)  SEMCAD,  b)  Fortran  and  c)  C.  All  three  plots  use  the  same  linear  color-scale  from  0  to  1 
W/kg. 


6.  Rat  model 


A  small  rat  model  (51  by  22  by  114  voxels)  was  used  as  the  final  model  in  the  comparison.  The 
model  was  run  at  5  different  frequencies.  The  results  are  summarized  in  Table  6.  Values  of  tissue 
SAR  were  also  compared,  and  they  agree  best  at  frequencies  from  300  to  500  MHz,  with  larger 
discrepancies  appearing  at  600  and  900  MHz.  At  frequencies  from  300  to  500  MHz  the  largest 
differences  in  tissue  mean  SAR  are  +30  and  -18  %,  with  the  average  difference  being  between  -1 
and  2  %.  At  900  MHz,  the  differences  increase  to  +444  and  -85  %  with  the  average  difference 
between  all  tissues  being  35  %. 


Frequency  [MHz]  Fortran  [W/kg]  C  [W/kg]  %  difference 


300 

0.275907 

0.281259 

2% 

400 

0.4765121 

0.475426 

0% 

500 

0.7734072 

0.78925 

2% 

600 

0.8268738 

0.90397 

9% 

900 

0.5041003 

0.594133 

18% 

Table  6:  Values  of  whole-body  SAR  In  the  rat  model 


Whole-body  SAR  [W/kg] 


Figure  23:  Whole-body  SAR  in  the  rat  model  versus  frequency 


7.  Conclusion 


The  results  show  great  consistency  between  the  two  different  implementations  of  FDTD  code 
across  a  wide  variety  of  models  and  exposure  scenarios.  There  are  some  minor  outliers  in  the  whole 
body  SAR  comparison,  and  some  larger  differences  in  the  localized  tissue  SAR  values.  However, 
these  can  be  explained  by  the  fact  that  although  the  model  and  exposure  details  were  the  same, 
the  criteria  for  finishing  a  simulation  (i.e.  the  ratio  between  two  successive  whole-body  SAR  values) 
can  be  satisfied  over  the  whole  body,  even  if  the  distribution  of  power  over  the  internal  organs  is 
not  entirely  stabilized  yet. 

The  values  would  converge  better  if  much  longer  simulations  (in  terms  of  number  of  halfcycles  of 
the  incident  field,  or  number  of  time-steps  of  the  Yee  algorithm)  were  used.  However,  when  tissue 
SAR  values  are  much  lower  than  the  basic  restrictions  on  localized  SAR,  the  discrepancies  in  the 
range  of  30  %  are  not  significant.  In  fact,  the  best  measurement  setups  for  measuring  localized  SAR 
values  in  homogeneous  liquid-filled  phantoms  have  a  measurement  uncertainty  around  1  dB 
(around  20  %),  so  differences  in  localized  tissue  values  in  this  range  actually  represent  a  very  good 
result. 

The  comparison  with  third  party  software  SEMCAD  also  showed,  that  the  results  are  very  good  and 
especially  whole-body  SAR  values  agreed  very  well,  considering  the  differences  in  voxeling, 
normalization  and  most  importantly  SAR  statistics  algorithms. 

Although  some  values  show  rather  large  differences,  this  could  also  be  in  part  due  to  different 
hardware  setups  that  the  results  were  obtained  on  or  due  to  different  approaches  to 
parallelization. 

The  conclusion  is,  that  the  two  different  implementations  of  the  FDTD  algorithm  produce  very 
similar  results,  to  the  point  of  being  indistinguishable. 
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